a critical examination and new developments

2012-2013Remote sensing consists in measuring some characteristics of an object from a distance. A key example of remote sensing is the Earth observation from sensors mounted on satellites that is a crucial aspect of space programs. The first satellite used for Earth observation was Explorer VII. It has been followed by thousands of satellites, many of which are still working. Due to the availability of a large number of different sensors and the subsequent huge amount of data collected, the idea of obtaining improved products by means of fusion algorithms is becoming more intriguing. Data fusion is often exploited for indicating the process of integrating multiple data and knowledge related to the same real-world scene into a consistent, accurate, and useful representation. This term is very generic and it includes different levels of fusion. This dissertation is focused on the low level data fusion, which consists in combining several sources of raw data. In this field, one of the most relevant scientific application is surely the Pansharpening. Pansharpening refers to the fusion of a panchromatic image (a single band that covers the visible and near infrared spectrum) and a multispectral/hyperspectral image (tens/hundreds bands) acquired on the same area. [edited by author]XII ciclo n.s

Band-wise Hyperspectral Image Pansharpening using CNN Model Propagation

Hyperspectral pansharpening is receiving a growing interest since the last few years as testified by a large number of research papers and challenges. It consists in a pixel-level fusion between a lower-resolution hyperspectral datacube and a higher-resolution single-band image, the panchromatic image, with the goal of providing a hyperspectral datacube at panchromatic resolution. Thanks to their powerful representational capabilities, deep learning models have succeeded to provide unprecedented results on many general purpose image processing tasks. However, when moving to domain specific problems, as in this case, the advantages with respect to traditional model-based approaches are much lesser clear-cut due to several contextual reasons. Scarcity of training data, lack of ground-truth, data shape variability, are some such factors that limit the generalization capacity of the state-of-the-art deep learning networks for hyperspectral pansharpening. To cope with these limitations, in this work we propose a new deep learning method which inherits a simple single-band unsupervised pansharpening model nested in a sequential band-wise adaptive scheme, where each band is pansharpened refining the model tuned on the preceding one. By doing so, a simple model is propagated along the wavelength dimension, adaptively and flexibly, with no need to have a fixed number of spectral bands, and, with no need to dispose of large, expensive and labeled training datasets. The proposed method achieves very good results on our datasets, outperforming both traditional and deep learning reference methods. The implementation of the proposed method can be found on https://github.com/giu-guarino/R-PN

Spatio-temporal resolution enhancement for cloudy thermal sequences

Many applications require remotely sensed brightness temperature (BT) data acquired with high temporal and spatial resolutions. In this regard, a viable strategy to overtake the physical limitations of space-borne sensors to achieve these data relies on fusing low temporal but high spatial resolution (HSR) data with high temporal but low spatial resolution data. The most promising methods rely on the fusion of spatially interpolated high temporal resolution data with temporally interpolated HSR data. However, the unavoidable presence of cloud masses in the acquired image sequences is often neglected, compromising the functionality and/or the effectiveness of the most of these fusion algorithms. To overcome this problem, a framework combining techniques of temporal smoothing and spatial enhancement is proposed to estimate surface BTs with high spatial and high temporal resolutions even when cloud masses corrupt the scene. Numerical results using real thermal data acquired by the SEVIRI sensor show the ability of the proposed approach to reach better performance than techniques based on either only interpolation or only spatial sharpening, even dealing with missing data due to the presence of cloud masses

Fusion of MultiSpectral and Panchromatic Images Based on Morphological Operators

International audienceNonlinear decomposition schemes constitute an alternative to classical approaches for facing the problem of data fusion. In this paper we discuss the application of this methodology to a popular remote sensing application called pansharpening, which consists in the fusion of a low resolution multispectral image and a high resolution panchromatic image. We design a complete pansharpening scheme based on the use of morphological half gradients operators and demonstrate the suitability of this algorithm through the comparison with state of the art approaches. Four datasets acquired by the Pleiades, Worldview-2, Ikonos and Geoeye-1 satellites are employed for the performance assessment, testifying the effectiveness of the proposed approach in producing top-class images with a setting independent of the specific sensor

A Benchmarking Protocol for SAR Colorization: From Regression to Deep Learning Approaches

Synthetic aperture radar (SAR) images are widely used in remote sensing. Interpreting SAR images can be challenging due to their intrinsic speckle noise and grayscale nature. To address this issue, SAR colorization has emerged as a research direction to colorize gray scale SAR images while preserving the original spatial information and radiometric information. However, this research field is still in its early stages, and many limitations can be highlighted. In this paper, we propose a full research line for supervised learning-based approaches to SAR colorization. Our approach includes a protocol for generating synthetic color SAR images, several baselines, and an effective method based on the conditional generative adversarial network (cGAN) for SAR colorization. We also propose numerical assessment metrics for the problem at hand. To our knowledge, this is the first attempt to propose a research line for SAR colorization that includes a protocol, a benchmark, and a complete performance evaluation. Our extensive tests demonstrate the effectiveness of our proposed cGAN-based network for SAR colorization. The code will be made publicly available.Comment: 16 pages, 16 figures, 6 table

Multispectral pansharpening with radiative transfer-based detail-injection modeling for preserving changes in vegetation cover

Whenever vegetated areas are monitored over time, phenological changes in land cover should be decoupled from changes in acquisition conditions, like atmospheric components, Sun and satellite heights and imaging instrument. This especially holds when the multispectral (MS) bands are sharpened for spatial resolution enhancement by means of a panchromatic (Pan) image of higher resolution, a process referred to as pansharpening. In this paper, we provide evidence that pansharpening of visible/near-infrared (VNIR) bands takes advantage of a correction of the path radiance term introduced by the atmosphere, during the fusion process. This holds whenever the fusion mechanism emulates the radiative transfer model ruling the acquisition of the Earth's surface from space, that is for methods exploiting a multiplicative, or contrast-based, injection model of spatial details extracted from the panchromatic (Pan) image into the interpolated multispectral (MS) bands. The path radiance should be estimated and subtracted from each band before the product by Pan is accomplished. Both empirical and model-based estimation techniques of MS path radiances are compared within the framework of optimized algorithms. Simulations carried out on two GeoEye-1 observations of the same agricultural landscape on different dates highlight that the de-hazing of MS before fusion is beneficial to an accurate detection of seasonal changes in the scene, as measured by the normalized differential vegetation index (NDVI)

Multi-resolution analysis techniques and nonlinear PCA for hybrid pansharpening applications

International audienceHyperspectral images have a higher spectral resolution (i.e., a larger number of bands covering the electromagnetic spectrum), but a lower spatial resolution with respect to multispectral or panchromatic acquisitions. For increasing the capabilities of the data in terms of utilization and interpretation, hyperspectral images having both high spectral and spatial resolution are desired. This can be achieved by combining the hyperspectral image with a high spatial resolution panchromatic image. These techniques are generally known as pansharpening and can be divided into component substitution (CS) and multi-resolution analysis (MRA) based methods. In general, the CS methods result in fused images having high spatial quality but the fused images suffer from spectral distortions. On the other hand, images obtained using MRA techniques are not as sharp as CS methods but they are spectrally consistent. Both substitution and filtering approaches are considered adequate when applied to multispectral and PAN images, but have many drawbacks when the low-resolution image is a hyperspectral image. Thus, one of the main challenges in hyperspectral pansharpening is to improve the spatial resolution while preserving as much as possible of the original spectral information. An effective solution to these problems has been found in the use of hybrid approaches, combining the better spatial information of CS and the more accurate spectral information of MRA techniques. In general, in a hybrid approach a CS technique is used to project the original data into a low dimensionality space. Thus, the PAN image is fused with one or more features by means of MRA approach. Finally the inverse projection is used to obtain the enhanced image in the original data space. These methods, permit to effectively enhance the spatial resolution of the hyperspectral image without relevant spectral distortions and on the same time to reduce the computational load of the entire process. In particular, in this paper we focus our attention on the use of Non-linear Principal Component Analysis (NLPCA) for the projection of the image into a low dimensionality feature space. However, if on one hand the NLPCA has been proved to better represent the intrinsic information of hyperspectral images in the feature space, on the other hand, an analysis of the impact of different fusion techniques applied to the nonlinear principal components in order to define the optimal framework for the hybrid pansharpening has not been carried out yet. More in particular, in this paper we analyze the overall impact of several widely used MRA pansharpening algorithms applied in the nonlinear feature space. The results obtained on both synthetic and real data demonstrate that, an accurate selection of the pansharpening method can lead to an effective improvement of the enhanced hyperspectral image in terms of spectral quality and spatial consistency, as well as a strong reduction in the computational time

Atmospheric boundary layer height estimation from aerosol lidar: A new approach based on morphological image processing techniques

The atmospheric boundary layer (ABL) represents the lowermost part of the atmosphere directly in contact with the Earth's surface. The estimation of its depth is of crucial importance in meteorology and for anthropogenic pollution studies. ABL height (ABLH) measurements are usually far from being adequate, both spatially and temporally. Thus, different remote sensing sources can be of great help in growing both the spatial and temporal ABLH measurement capabilities. To this aim, aerosol backscatter profiles are widely used as a proxy to retrieve the ABLH. Hence, the scientific community is making remarkable efforts in developing automatic ABLH retrieval algorithms applied to lidar observations. In this paper, we propose a ABLH estimation algorithm based on image processing techniques applied to the composite image of the total attenuated backscatter coefficient. A pre-processing step is applied to the composite total backscatter image based on morphological filters to properly set-up and adjust the image to detect edges. As final step, the detected edges are post-processed through both mathematical morphology and an object-based analysis. The performance of the proposed approach is assessed on real data acquired by two different lidar systems, deployed in Potenza (Italy) and Évora (Portugal), belonging to the European Aerosol Research Lidar Network (EARLINET). The proposed approach has shown higher performance than the benchmark consisting of some state-of-The-Art ABLH estimation methods. © 2021 Copernicus GmbH. All rights reserved.ACTRIS (https://www.actris.eu/, last access: 15 March 2021) has received funding from the European Union’s Horizon 2020 research and innovation program under grant agreement nos. 654109 (ACTRIS-2), 759530 (ACTRIS-PPP), 871115 (ACTRIS-IMP) and 824068 (ENVRI-FAIR), and previously from the European Union Seventh Framework Programme (FP7/2007-2013) under grant agreement no. 262254. The Portuguese lidar station is also supported by national funds through FCT – Foundation for Science and Technology, I. P., within the scope of projects UIDB/04683/2020 and UIDP/04683/2020, and also through project TOMAQAPA (PTDC/CTAMET/29678/2017). Moreover, the authors gratefully acknowledge CloudNET for providing ECMWF and GDAS atmospheric forecasts for all the measurement cases included in this study